The invention relates to technology for verifying designs and for analysis of coverage information for verifying designs.
Many phases of modern design activities are performed with computer aided design (CAD) tools or electronic design automation (EDA) systems. For example, for the process of creating an electronic design (e.g., of an integrated circuit or “IC”), a designer first creates high level behavior descriptions of the IC device using a high-level hardware design language (HDL). Common examples of HDLs include Verilog and VHDL. An EDA system typically receives the high level behavior descriptions of the IC device and translates this high-level design language into netlists of various levels of abstraction using a computer synthesis process. A netlist describes interconnections of nodes and components on the chip and includes information of circuit primitives such as transistors and diodes, along with their sizes and interconnections. An integrated circuit designer may use a set of layout EDA application programs to create a physical integrated circuit design layout from a logical circuit design. The layout EDA application uses geometric shapes of different materials to create the various electrical components on an integrated circuit and to represent electronic and circuit IC components as geometric objects with varying shapes and sizes. CAD or EDA tools may also be used to generate software designs, system level designs, or mixed hardware-software designs.
Modern designs are often very large and complex. For example, a modern circuit design may have many millions of transistors and a modern software design may have many millions of lines of code. As designs become more complex, the likelihood of design errors increases. Therefore, it is very likely that design tools are also needed to manage the complexity of the design, particularly to verify that the design will work for its intended purpose. For example, with the increasing complexity of digital designs, functional verification is beginning to occupy a significant portion of the overall design cycle and effort of the design process for an integrated circuit design.
Design verification is the process of determining whether a design, such as an integrated circuit, board, software, or system-level architecture, adequately implements the requirements defined by the specification of the architecture for that design. Design verification for a design or device under testing (DUT) may be performed on the actual device/software or on a simulation model of the device/software. However, design verification is becoming more difficult and time consuming as the underlying designs themselves are becoming larger and more complex, and the simulation models of the design are also becoming more complex to prepare and to test. For the purposes of explanation only and without intending to be limiting in any way, the following discussion centers upon verification/testing activities which are performed on simulation models of the device. As used herein and without intent of limitation, the terms “testing” and “verification” are used interchangeably.
To verify a design, the simulated model of the design can receive input stimuli in the form of test vectors (e.g., as a string of binary digits applied to the input of a circuit). The simulated model then produces results, which are checked against the expected results for the particular design of the device. For example, the test generator disclosed in U.S. Pat. No. 6,182,258 can be used to generate tests to interact with a design. The test generation procedure is written in a verification specific object-oriented programming language and is used to write various tests which automatically create a device verification test by a test generator module. A mix of directed, random and constrained-random techniques can be employed to perform simulation-based functional verification.
However, it is important that there is adequate “coverage” of the portions of the design that needs to be tested. Coverage is a function of the ability to fully explore the reachable state space of the design, in order to examine possible functions and outcomes of any particular input or set of inputs to the DUT. With modern designs becoming extremely large, it is becoming more and more difficult to confirm adequate coverage of the tests. For purposes of illustration, and not by way of limitation, the examples in this document explicitly address the hardware verification domain, but the concepts described herein are applicable as well to software verification or any other domain which employs coverage analysis.
Coverage analysis is the process of analyzing the coverage of a given test or set of test activities, and is often applied to multiple verification domains, for example to software verification or hardware verification. Verification coverage can be broadly divided into code coverage and functional coverage. Code coverage is derived from the code (e.g., HDL code) for the design being verified. Functional coverage is specified explicitly by means of a number of functional verification constructs, e.g., using assertion language constructs such as Accellera PSL (property specification language) and SVA (System Verilog Assertions) cover constructs, SystemVerilog/e covergroups, and/or transactions. Code coverage may also includes FSM (finite state machine) coverage, which helps interpret the synthesis semantics of the HDL design and monitors the coverage of the FSM representation of control logic in the design.
Analyzing the coverage to identify areas of improvement is an important step of coverage driven verification after coverage data has been collected from simulation-based verification runs. The analysis is followed by additional actions to improve the coverage of the test. In many cases, the coverage analysis is performed on hierarchical structures, where the hierarchy used for the analysis is the design hierarchy itself. This correspondence between the hierarchy and the test coverage allows the user to relate the coverage to the design objects. For plan-based verification methodologies, the user performs analysis using the hierarchy of a verification plan, which is a structured document containing a hierarchy of sections. The hierarchical structure may be a design hierarchy, in which the basic element or node of the hierarchy is a design instance. The design hierarchy may also be a user-defined section hierarchy for a verification plan, in which the basic element or node of this hierarchy is a section.
A graphical coverage analysis tool provides multiple features to analyze the coverage data in hierarchical structures. The graphical presentation of hierarchical coverage information should allow quick browsing of hierarchies along with fast identification of hierarchical nodes which contribute low coverage. The primary motive is to find an efficient representation of hierarchical structure to perform the coverage analysis, where reaching coverage of 100% in all basic elements of a hierarchy implies 100% verification completion. Therefore, from a practical point of view, the goal is to reach 100% cumulative-coverage for the top level root of the hierarchy. This means that all sub elements have 100% self-coverage as well. However, achieving the figure of 100% is not easy, and requires deep understanding of the missing coverage, followed by some directed steps to enhance it.
FIG. 1 shows a conventional approach for implementing a user interface for analyzing verification coverage information in a hierarchy. The approach of FIG. 1 includes a tree browser 102 that displays the hierarchy in a node-and-list representation. When used for coverage analysis, this node-and-list representation shows the coverage for each node in the list. The user can scroll through the hierarchical tree, and expand or contract the sub-node details, to analyze its coverage. The details pane 106 displays the complete details for each selected node in the tree browser representation of the hierarchy. The sub-elements table 104 shows the complete list of sub-elements of a tree underneath a node. This type of interface provides information about the coverage details for a specific node, as well as the location and details of elements in the sub-elements table.
A significant problem with this approach, however, is that display of a large list of elements in the hierarchy browser 102 can overwhelm the available display space. In other words, a large list of nodes will be too large to allow all of the nodes to fit on the screen at the same time, which means that the user must scroll up and down or page forward and backward through the screen to view the node details. In addition, to maximize the fit of the list onto the screen, some or all of the sub-nodes must be collapsed and therefore not directly displayed to the user. These deficiencies can severely affect the user's ability to meaningfully review and analyze the hierarchy, and negatively affect the user's ability to grasp the entire picture of the hierarchical structure. Additionally, this approach is inefficient since it requires the user to scroll through the complete list of nodes and individually look at their coverage in-order to make a decision as to which nodes are contributing to low overall cumulative-coverage of the design. As the levels of hierarchy and number of nodes increase, the space occupied by the tree representation increases proportionately.